Feat: add configurable forecast post processing#2273
Conversation
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Documentation build overview
15 files changed ·
|
| "0 kW": [ | ||
| "0 kW", | ||
| "4 kW" | ||
| ] |
There was a problem hiding this comment.
I didn't think of this before, but there is an edge case we did not think of:
If the forecast was exactly 4 kW is it also snapped to 0 kW or not? My first instinct was to have it not snap, i.e. the interval excludes the boundaries. However, that opens up a secondary edge case:
{
"0 kW": [
"0 kW",
"4 kW"
],
"10 kW": [
"4 kW",
"10 kW"
]
}
Does 4 kW now snap to 0 kW or 10 kW?
Actually, if the interval does include the boundaries, the same question applies.
Let's discuss this.
There was a problem hiding this comment.
Yes indeed, this is an interesting edge case to guard against , I thought about it and have three suggestions:
- For adjacent intervals, interpret the shared boundary as belonging to the interval that starts there.
For example:
{
"0 kW": ["0 kW", "4 kW"],
"10 kW": ["4 kW", "10 kW"],
"11 kW": ["13 kW", "15 kW"],
"12 kW": ["15 kW", "16 kW"]
}
would be interpreted as:
[0, 4)
[4, 10]
[13, 15)
[15, 16]
So exactly 4 kW snaps to 10 kW, exactly 10 kW still snaps to 10 kW, exactly 15 kW snaps to 12 kW, and exactly 16 kW still snaps to 12 kW.
- Reject shared boundaries as invalid config.
Most simple solution, so this would raise an error:
{
"0 kW": ["0 kW", "4 kW"],
"10 kW": ["4 kW", "10 kW"]
}
because exactly 4 kW has two possible snap targets.
- Make the boundary behavior explicit with extra fields.
For example:
{
"0 kW": {
"lower": "0 kW",
"upper": "4 kW",
"include_lower": true,
"include_upper": false
}
}
This is the most explicit, but it makes the config too heavy maybe.
I like option 1 best if we document it clearly, because it keeps the config simple and gives adjacent intervals an intuitive behavior.
There was a problem hiding this comment.
I like option 1, too, but worth a slight altercation.
But before I explain the altercation, first a clarification.
In
{
"0 kW": ["0 kW", "4 kW"],
"10 kW": ["4 kW", "10 kW"],
"11 kW": ["13 kW", "15 kW"],
"12 kW": ["15 kW", "16 kW"]
}
In a traditional snap, the values would snap to one of the boundaries. Here, values between 13 and 15 snap to 11, which is outside of the range. Same for values between 15 and 16 snapping to 12. We should probably stick to traditional snapping and enforce that the key is one of the boundaries.
So I'll now switch to a more "traditional" example:
{
"0 kW": ["0 kW", "4 kW"],
"10 kW": ["4 kW", "10 kW"],
"13 kW": ["13 kW", "15 kW"],
"15 kW": ["15 kW", "16 kW"]
}
Then my altercation:
I suggest interpreting the first boundary as inclusive and the second as exclusive, and then allowing users to specify the interval in reverse order.
Then:
{
"0 kW": ["0 kW", "4 kW"],
"10 kW": ["4 kW", "10 kW"],
"13 kW": ["13 kW", "15 kW"],
"15 kW": ["15 kW", "16 kW"]
}
becomes [0, 4), [4, 10), [13, 15) and [15, 16). The difference to before being that forecasts of exactly 10 or 16 are not snapped down.
If you still want to snap them, you'd pass:
{
"0 kW": ["0 kW", "4 kW"],
"10 kW": ["10 kW", "4 kW"],
"13 kW": ["13 kW", "15 kW"],
"15 kW": ["16 kW", "15 kW"]
}
which would become [0, 4), (4, 10], [13, 15) and (15, 16]. Note that for the number used for the key (where it snaps to) it doesn't matter whether that is in the first or second position, because it would just snap to itself.
This does rely on a good field description, though.
Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Signed-off-by: Mohamed Belhsan Hmida <149331360+BelhsanHmida@users.noreply.github.com>
Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Signed-off-by: Mohamed Belhsan Hmida <149331360+BelhsanHmida@users.noreply.github.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Description
lower: clip forecast values below this boundupper: clip forecast values above this boundsnap: replace forecast values inside configured intervals with a target valueensure-positivebehavior unchanged for backwards compatibility.How to test
Automated tests:
Manual test:
/tmp/forecast-post-processing.yml:kWsensor with enough historical data:0 kWare stored as0 kW20 kWare stored as20 kW0 kWand4 kWare stored as0 kWAlternative unit-conversion check:
Expected result: forecasts above
20 kWare clipped to20 kW.Unitless snap check:
If the output sensor unit is
kW, this config:is interpreted as:
I manually tested this with a temporary
kWsensor trained on constant2.5 kWvalues. The stored forecast values were:Further Improvements
lower <= uppercurrently happens when post-processing runs. We may move this earlier into schema/parameter validation.Related Items
Closes #2270
Sign-off